Coordinated action robotic system and related methods

ABSTRACT

A coordinated action robotic system may include a plurality of robotic vehicles, each including a platform and at least one manipulator movable relative thereto. The robotic system may also include a remote operator control station that may include a respective controller for each manipulator. The remote operator control station may also include a mapping module to map movement of each manipulator relative to its platform. Operation of the controllers for manipulator movement in a given direction produces corresponding movement of the respective manipulators in the given direction such that the robotic vehicles may be controlled as if they were one robotic vehicle. The coordinated movement may result in increased operational efficiency, increased operational dexterity, and increased ease of controlling the robotic vehicles.

FIELD OF THE INVENTION

The present invention relates to the field of robotic vehicles, and moreparticularly, coordinated control of robotic vehicles and relatedmethods.

BACKGROUND OF THE INVENTION

A robot may be used for a variety of unmanned operations. One example ofan unmanned robot is an unmanned ground vehicle (UGV), for example, theiRobot Packbot, available from iRobot of Bedford, Mass. A UGV maytypically include a platform, and a manipulator carried by the platformthat is often primitive and includes joint level control. The singlemanipulator often performs all of the operations.

A commercial-off-the-shelf robot may have increased flexibility and mayinclude dual arm, dexterous manipulators. One particular example is theMotoMan DIA 10, available from Motoman, Inc. of West Carrollton, Ohio.Academia and research groups may also provide robots with increasedfunctionality, for example, the NASA Robonaut.

U.S. Pat. No. 6,898,484 to Lemelson et al. discloses a system forcontrolling manufacturing operations. A location of a target objectrelative to a robotic manipulator is input into a control system. Themanipulator and target object are located and tracked via the globalpositioning system. The control system directs the manipulator toperform operations on each target object based upon the location of themanipulator and target object.

Reduced cost and reduced weight may correspond to limited robotfunctions. Control of a robot may be limited by a number of ways therobot may be manipulated. More particularly, for example, a robot mayhave limited dexterity and limited movement in a finite number ofdirections as a result.

Moreover, when more than one robot having limited dexterity and limitedmovement is used, it may be difficult for an operator to control eachrobot. More particularly, when each robot's platform is positioned in adifferent direction, an operator may find it increasingly difficult tocontrol respective manipulators in a common direction or havecoordinated movements from a controller correspond to coordinatemovements of the respective manipulators.

SUMMARY OF THE INVENTION

In view of the foregoing background, it is therefore an object of thepresent invention to provide coordinated robotic vehicle manipulatormovements.

This and other objects, features, and advantages in accordance with thepresent invention are provided by a coordinated action robotic systemthat includes a plurality of robotic vehicles. Each of the plurality ofrobotic vehicles may include a platform and at least one manipulatormovable relative thereto. The robotic system may also include a remoteoperator control station that may include a respective controller foreach manipulator, for example. The remote operator control station mayfurther include a mapping module to map movement of each manipulatorrelative to its platform so that operation of the controllers formanipulator movement in a given direction produces correspondingmovement of the respective manipulators in the given direction.Accordingly, the coordinated action robotic system provides coordinatedrobotic vehicle manipulator movements, and thus increased dexteritythereof.

The mapping module may map movement of each manipulator based upon acommon coordinate system. The common coordinate system may be based upona selected one of the robotic vehicles, for example.

Each robotic vehicle may include a sensor arrangement for sensing arelative position and orientation between the platform and the at leastone manipulator. The mapping module may map movement of each manipulatorbased upon the relative position and orientation between each platformand at least one manipulator, for example. The sensor arrangement mayalso sense yaw, heading, and attitude of the platform.

Each robotic vehicle may further include a geospatial positiondetermining device. The mapping module may map movement of eachmanipulator based upon the geospatial position of each robotic vehicle.

The robotic system may further include a respective communications linkbetween each robotic vehicle and the remote operator control station. Atleast one of the communications links may include a wirelesscommunications link, for example.

Each robotic vehicle may further include at least one image sensorcarried by the platform. The remote operator control station may furtherinclude at least one display for displaying images from the imagesensors. Each robotic vehicle may include a ground drive arrangementcarried by the platform, for example.

A method aspect may include coordinating robotic action in a roboticsystem. The robotic system may include a plurality of robotic vehicles,each including a platform and at least one manipulator movable relativethereto, and a remote operator control station including a respectivecontroller for each manipulator, for example. The method may includemapping movement, using the remote operator control station, of eachmanipulator relative to its platform so that operation of thecontrollers for manipulator movement in a given direction producescorresponding movement of the respective manipulators in the givendirection.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram of a coordinated action robotic system inaccordance with the present invention.

FIG. 2 is a more detailed schematic block diagram of a coordinatedaction robotic system of FIG. 1.

FIG. 3 is a perspective view illustrating the operation of the mappingmodule at the robotic vehicles of FIG. 2.

FIG. 4 is a schematic plan view illustrating mapped movement of thesystem of FIG. 2.

FIG. 5 is a flow chart illustrating a method of coordinating roboticaction in accordance with the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present invention will now be described more fully hereinafter withreference to the accompanying drawings, in which preferred embodimentsof the invention are shown. This invention may, however, be embodied inmany different forms and should not be construed as limited to theembodiments set forth herein. Rather, these embodiments are provided sothat this disclosure will be thorough and complete, and will fullyconvey the scope of the invention to those skilled in the art. Likenumbers refer to like elements throughout, and prime notation is used toindicate similar elements in alternative embodiments.

Referring initially to FIGS. 1 and 2, a coordinated action roboticsystem 10 illustratively includes a plurality of robotic vehicles 12a-12 n. Each of the plurality of robotic vehicles 12 a-12 n includes aplatform 15 a-15 n and a manipulator 13 a-13 n movable relative thereto.As will be appreciated by those skilled in the art, each robotic vehicle12 a-12 n may include more than one manipulator 13 a-13 n, each moveablerelative to the corresponding platform 15 a-15 n. Each manipulator 13a-13 n may be, for example, a robotic arm that is capable of graspingand moving an object. Other types of manipulators may be used.

Each robotic vehicle 12 a-12 n also includes a ground drive arrangement24 a-24 n carried by the platform 15 a-15 n. Illustratively, the grounddrive arrangement 24 a-24 n includes a pair of individually driventracks for good maneuverability in different terrain. Other types ofground drive arrangements may be used, such as a wheel-based grounddrive arrangement, as will be appreciated by those skilled in the art.Of course, the platform drive arrangement may permit operation on wateror other surfaces as well.

The robotic system 10 also illustratively includes a remote operatorcontrol station 30. The remote operator control station 30 includes arespective controller 31 a-31 n for each manipulator 13 a-13 n. Eachrespective controller 31 a-31 n, for example, as illustrated, may be ajoystick. Other respective controllers (not shown) may also be used forcontrolling locomotion of each robotic vehicle 12 a-12 n. Other types ofcontrollers may be used, and controller functionality may be shared, aswill be appreciated by those skilled in the art.

Referring more particularly to FIG. 2, each robotic vehicle 12 a-12 nfurther illustratively includes an image sensor 22 a-22 n carried by theplatform 15 a-15 n. For example, the image sensor 22 a-22 n may be acamera to advantageously provide an operator at the remote operationcontrol station 30 a visual indication of the environment of eachrobotic vehicle 12 a-12 n. Other types of image sensors may be used,such as in infrared or night vision, as will be appreciated by thoseskilled in the art.

Each robotic vehicle 12 a-12 n also includes a sensor arrangement 16 forsensing a relative position and orientation between the platform and themanipulator 13 a-13 n. The sensor arrangement 16 may also cooperate witha robotic vehicle processor 25 to sense, for example, yaw, heading, andattitude of the platform 15 a-15 n. In some embodiments, yaw, heading,and attitude data may be provided by an Attitude Heading ReferenceSystem (AHRS). Yaw, heading, and attitude may be provided by other typesof sensor arrangements, as will be appreciated by those skilled in theart.

Each robotic vehicle 12 a-12 n illustratively includes a geospatialposition determining device 17. For example, each robotic vehicle 12a-12 n may include a global positioning system (GPS) receiver 17 fordetermining geospatial position from a GPS satellite 11.

The sensed relative position and orientation between the platform 15a-15 n and the manipulator 13 a-13 n, along with the yaw, heading andattitude data, may be communicated to the remote operator controlstation 30 via the respective robotic vehicle transceiver 21 a-21 n. Therobotic vehicle transceiver 21 a-21 n may communicate with the remoteoperator control station transceiver 33 via a respective communicationslink 23 a-23 n between each robotic vehicle 12 a-12 n and the remoteoperator control station.

Each respective communications link 23 a-23 n illustratively includes awireless communications link. For example, the wireless communicationslink may be an RF communications link. In other embodiments, some or allof the respective communications links 23 a-23 n may be wiredcommunications links, for example, as provided by tether assemblies.Other wired or wireless communications links may be used, as will beappreciated by those skilled in the art.

The geospatial position data may be similarly communicated to the remoteoperator control station 30, and may be communicated over the respectivecommunications links 23 a-23 n. Additional communications links (notshown) may be provided for communicating the geospatial position data.Other data, for example, robotic vehicle locomotion commands from theremote operator control station 30 and image sensor data or imagesassociated with each image sensor 22 a-22 n, may also be communicatedover each communications link 23 a-23 n. Images associated with eachimage sensor 22 a-22 n are displayed on a display 34 included at theremote operator control station 30. Other robotic vehicle sensors,including state-of-health and power supply sensors may also be included.

The remote operator control station 30 includes a processor 35 that, inturn, includes a mapping module 32, to map movement of each manipulator13 a-13 n relative to its platform 15 a-15 n so that operation of thecontrollers 31 a-31 n for manipulator movement in a given directionproduces corresponding movement of the respective manipulators in thegiven direction.

Based upon the relative position and orientation between each platform15 a-15 n and its corresponding manipulator 13 a-13 n received by theremote operator control station transceiver 33, the mapping module 32maps movement of each manipulator. The mapping module 32 mayadditionally geospatially map movement of each manipulator 13 a-13 nbased upon the received geospatial position of each robotic vehicle 12a-12 n.

The mapping module 32 advantageously maps movement of each manipulator13 a-13 n based upon a common coordinate system. As will be appreciatedby those skilled in the art, the common coordinate system may be basedupon a selected one of any of the robotic vehicles 13 a-13 n. In someembodiments, the common coordinate system may be based upon a targetobject 14, or may be based upon any other defined coordinate system.

Referring now additionally to FIG. 3, an example of the operation of themapping module 32 is now described. Illustratively, robotic vehicles 12a, 12 b each have a different orientation with reference to the worldcoordinate system xyz_(w). An AHRS 16 a, 16 b, for example, provides theglobal orientation, denoted by xyz_(A) and xyz_(B), of the roboticvehicles 12 a, 12 b with respect to the world coordinate system xyz_(w).The mapping module 32 defines the elements of the transforms T_(A,arm)^(A) and T_(B,arm) ^(B), respectively. Transforms T_(A,arm) ^(A) andT_(B,arm) ^(B), represent transformation matrices to the respectiveplatforms 15 a, 15 b of each manipulator 13 a, 13 b with respect to therespective coordinate systems at the vehicles 12 a, 12 b.

Based upon the above orientation information, including the transforms,the mapping module 32 calculates a unit vector pointing from xyz_(A,arm)to xyz_(B,arm.)

The unit vector is illustratively represented by {right arrow over(x)}′_(arms) and {right arrow over (g)} represents the unit vectorpointing in the direction of gravity, as measured by the AHRS 16 a, 16b.

A {right arrow over (y)}_(arms) vector can be formed by crossing {rightarrow over (x)}′_(arms) with {right arrow over (g)}, for example, {rightarrow over (y)}_(arms)={right arrow over (x)}′_(arms)×{right arrow over(g)}. Similarly, {right arrow over (x)}_(arms) and {right arrow over(z)}_(arms) can be formed according to the following: {right arrow over(x)}_(arms)={right arrow over (g)}×{right arrow over (y)}_(arms), {rightarrow over (z)}_(arms)=−{right arrow over (g)}. The mapping module 32can compute the coordinate system xyz_(arms) and advantageously updateit on-the-fly to account for movement of each robotic vehicle 12 a, 12 bwith respect to the world coordinate system xyz_(w). The commoncoordinate system xyz_(arms) is illustratively referenced at themidpoint between xyz_(A,arm), and xyz_(B,arm), respectively. Motioncommands from the controllers 31 a, 31 b are performed in the commoncoordinate system xyz_(arms). In other words, input to each of thecontrollers 31 a, 31 b is mapped in the common coordinate systemxyz_(arms).

Referring now additionally to FIG. 4, in another embodiment, the commoncoordinate system is illustratively based upon a selected roboticvehicle 12 a′. Illustratively, the selected robotic vehicle 12 a′ has adifferent orientation with reference to the target object 14′. Amovement of the controller 31 a′, or joystick, in a direction indicatedby the arrow 26 a′ corresponds to movement of the manipulator 13 a′ in acorresponding direction indicated by the arrow 27 a′. Advantageously,the mapping module 32′ maps the movement of the manipulator 13 a′ basedupon the common coordinate system associated with the robotic vehicle 12a′. Accordingly, a movement of the controller 31 b′ in a same directionas the controller 31 a′, as indicated by the arrow 26 b′, corresponds tomovement of the manipulator 13 b′ in the same direction, as indicated bythe arrow 27 b′. As will be appreciated by those skilled in the art, amovement of the controller 31 b′ in a direction opposite to the movementof the controller 31 a′, corresponds to a movement of the manipulator 13b′ in a direction opposite to the direction of the manipulator 13 a′.

Advantageously, the mapping of the movement of each manipulator 13 a, 13b relative to its platform 15 a, 15 b to a common coordinate system sothat operation of the controllers 31 a, 31 b for manipulator movement ina given direction produces corresponding movement of the respectivemanipulators in the given direction allows the robotic vehicles 12 a, 12b to be controlled as if they were one robotic vehicle. Indeed, thecoordinated movement may result in increased operational efficiency,increased operational dexterity, and increased ease of controlling therobotic vehicles 12 a, 12 b.

As will be appreciated by those skilled in the art, any number ofadditional robotic vehicles may be included in the coordinated actionrobotic system 10 described above. Where more than two robotic vehiclesare included in the coordinated action robotic system 10, a commoncoordinate system is created and the movement of each manipulator 13 a,13 b, in other words, controller input commands, are mapped to thecommon coordinate system, as described above.

Referring now to the flowchart 50 of FIG. 5, a method aspect includescoordinating robotic action in a robotic system 10 including a pluralityof robotic vehicles 12 a-12 n. Each robotic system 10 includes aplatform 15 a-15 n and a manipulator 13 a-13 n movable relative thereto.Each robotic system 10 also includes a remote operator control station30 that includes a respective controller 31 a-31 n for each manipulator13 a-13 n. Beginning at Block 52, the method includes sensing ageospatial location at Block 54 determined by a geospatial positiondetermining device on each robotic vehicle. At Block 56, a relativeposition and orientation between each platform 15 a-15 n and therespective manipulator 13 a-13 n are sensed by a sensor 16 arrangementon each robotic vehicle 12 a-12 n. The sensor arrangement 16 also sensesyaw, heading, and attitude of each respective platform 15 a-15 n.

The sensed relative position and sensed orientation between eachplatform 15 a-15 n and the respective manipulator 13 a-13 n, along withthe geospatial position are communicated over each respectivecommunications link 23 a-23 n to the remote operator control station atBlock 58. At Block 60 movement of each manipulator 13 a-13 n relative toits platform 15 a-15 n is mapped to a common coordinate system. Mappingresults in operation of the controllers 31 a-31 n for manipulatormovement in a given direction producing corresponding movement of therespective manipulators 13 a-23 n in the given direction before endingat Block 62. In some embodiments, for example, as described above withreference to FIG. 4, the common coordinate system may be based upon aselected one of the robotic vehicles.

Many modifications and other embodiments of the invention will come tothe mind of one skilled in the art having the benefit of the teachingspresented in the foregoing descriptions and the associated drawings.Therefore, it is understood that the invention is not to be limited tothe specific embodiments disclosed, and that modifications andembodiments are intended to be included within the scope of the appendedclaims.

1. A coordinated action robotic system comprising: a plurality ofrobotic vehicles, each comprising a platform and at least onemanipulator movable relative thereto; and a remote operator controlstation comprising a respective controller for each manipulator, and amapping module to map movement of each manipulator relative to itsplatform so that operation of said controllers for manipulator movementin a given direction produces corresponding movement of the respectivemanipulators in the given direction.
 2. The robotic system according toclaim 1 wherein said mapping module maps movement of each manipulatorbased upon a common coordinate system.
 3. The robotic system accordingto claim 2 wherein the common coordinate system is based upon a selectedone of said plurality of robotic vehicles.
 4. The robotic systemaccording to claim 1 wherein each of said plurality of robotic vehiclescomprises a sensor arrangement for sensing a relative position andorientation between said platform and said at least one manipulator; andwherein said mapping module maps movement of each manipulator based uponthe relative position and orientation between each platform and at leastone manipulator.
 5. The robotic system according to claim 4 wherein saidsensor arrangement further senses yaw, heading, and attitude of saidplatform.
 6. The robotic system according to claim 1 wherein each ofsaid plurality of robotic vehicles further comprises a geospatialposition determining device; and wherein said mapping module mapsmovement of each manipulator based upon the geospatial position of eachrobotic vehicle.
 7. The robotic system according to claim 1 furthercomprising a respective communications link between each robotic vehicleand said remote operator control station.
 8. The robotic systemaccording to claim 7 wherein at least one of said communications linkscomprises a wireless communications link.
 9. The robotic systemaccording to claim 1 wherein each of said plurality of robotic vehiclesfurther comprises at least one image sensor carried by said platform;and wherein said remote operator control station further comprises atleast one display for displaying images from said image sensors.
 10. Therobotic system according to claim 1 wherein each of said plurality ofrobotic vehicles further comprises a ground drive arrangement carried bysaid platform.
 11. A coordinated action robotic system comprising: aplurality of robotic vehicles, each comprising a platform, at least onemanipulator movable relative to said platform, and a sensor arrangementfor sensing a relative position and orientation between said platformand said at least one manipulator; and a remote operator control stationcomprising a respective controller for each manipulator, and a mappingmodule to map to a common coordinate system, movement of eachmanipulator based upon the relative position and orientation betweeneach platform and at least one manipulator so that operation of saidcontrollers for manipulator movement in a given direction producescorresponding movement of the respective manipulators in the givendirection.
 12. The robotic system according to claim 11 wherein thecommon coordinate system is based upon a selected one of said pluralityof robotic vehicles.
 13. The robotic system according to claim 11wherein said sensor arrangement further senses yaw, heading, andattitude of said platform.
 14. The robotic system according to claim 11wherein each of said plurality of robotic vehicles further comprises ageospatial position determining device; and wherein said mapping modulemaps movement of each manipulator based upon the geospatial position ofeach robotic vehicle.
 15. The robotic system according to claim 11further comprising a respective communications link between each roboticvehicle and said remote operator control station.
 16. The robotic systemaccording to claim 15 wherein at least one of said communications linkscomprises a wireless communications link.
 17. The robotic systemaccording to claim 11 wherein each of said plurality of robotic vehiclesfurther comprises at least one image sensor carried by said platform;and wherein said remote operator control station further comprises atleast one display for displaying images from said image sensors.
 18. Therobotic system according to claim 11 wherein each of said plurality ofrobotic vehicles further comprises a ground drive arrangement carried bysaid platform.
 19. A method of coordinating robotic action in a roboticsystem comprising a plurality of robotic vehicles, each comprising aplatform and at least one manipulator movable relative thereto, and aremote operator control station comprising a respective controller foreach manipulator, the method comprising: mapping movement using theremote operator control station of each manipulator relative to itsplatform so that operation of the controllers for manipulator movementin a given direction produces corresponding movement of the respectivemanipulators in the given direction.
 20. The method according to claim19 wherein mapping movement of each manipulator comprises mappingmovement based upon a common coordinate system.
 21. The method accordingto claim 20 wherein the common coordinate system is based upon aselected one of the plurality of robotic vehicles.
 22. The methodaccording to claim 19 wherein mapping movement of each manipulatorcomprises mapping movement of each manipulator based upon a relativeposition and orientation between each platform and at least onemanipulator sensed by a sensor arrangement on each of the plurality ofrobotic vehicles.
 23. The method according to claim 22 wherein thesensor arrangement further senses yaw, heading, and attitude of theplatform.
 24. The method according to claim 19 wherein mapping movementof each manipulator comprises mapping movement based upon the geospatialposition of each of the plurality of robotic vehicles determined by ageospatial position determining device on each of the plurality ofrobotic vehicles.